Skip to content

WIP 🌱 chore(ci): Add test to check if user changes are preserved#2512

Open
camilamacedo86 wants to merge 1 commit intooperator-framework:mainfrom
camilamacedo86:test-asked
Open

WIP 🌱 chore(ci): Add test to check if user changes are preserved#2512
camilamacedo86 wants to merge 1 commit intooperator-framework:mainfrom
camilamacedo86:test-asked

Conversation

@camilamacedo86
Copy link
Contributor

@camilamacedo86 camilamacedo86 commented Feb 16, 2026

Add a test to ensure that OLM is not reverting user changes like kubectl rollout restart.

Copilot AI review requested due to automatic review settings February 16, 2026 17:56
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 16, 2026
@netlify
Copy link

netlify bot commented Feb 16, 2026

Deploy Preview for olmv1 ready!

Name Link
🔨 Latest commit f7e96d5
🔍 Latest deploy log https://app.netlify.com/projects/olmv1/deploys/69aee6468fea4d00086d6b95
😎 Deploy Preview https://deploy-preview-2512--olmv1.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@openshift-ci
Copy link

openshift-ci bot commented Feb 16, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign grokspawn for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds an end-to-end test to verify that OLMv1, when using Server-Side Apply, does not revert user-initiated changes to deployed resources. The test specifically validates the scenario where a user runs kubectl rollout restart deployment, which was problematic in OLMv0. The PR is explicitly marked as Work In Progress (WIP).

Changes:

  • Added new feature file rollout-restart.feature with a scenario testing user-initiated deployment changes
  • Implemented three new step functions: UserAddsRestartAnnotation, ResourceHasRestartAnnotation, and ClusterExtensionAnnotationIsSet

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
test/e2e/features/rollout-restart.feature New feature file defining the test scenario for verifying OLMv1 doesn't revert user-initiated changes to deployments
test/e2e/steps/steps.go Added three new step functions to support the rollout restart test scenario and registered them in the step definitions

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings February 18, 2026 09:13
@camilamacedo86 camilamacedo86 changed the title WIP: Add test verifies that OLMv1 does not revert user-initiated changes to deployed resources 🐛 fix: Preserve user changes to deployment pod templates Feb 18, 2026
@camilamacedo86 camilamacedo86 changed the title 🐛 fix: Preserve user changes to deployment pod templates 🐛 Preserve user changes to deployment pod templates Feb 18, 2026
@camilamacedo86 camilamacedo86 removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 18, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@camilamacedo86 camilamacedo86 changed the title 🐛 Preserve user changes to deployment pod templates WIP 🐛 Preserve user changes to deployment pod templates Feb 18, 2026
Copilot AI review requested due to automatic review settings February 18, 2026 15:04
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings February 26, 2026 17:34
@camilamacedo86 camilamacedo86 changed the title WIP 🐛 Preserve user changes to deployment pod templates chore(ci): Add test to check if user changes are preserved Feb 26, 2026
@camilamacedo86 camilamacedo86 changed the title chore(ci): Add test to check if user changes are preserved 🌱 chore(ci): Add test to check if user changes are preserved Feb 26, 2026
@camilamacedo86 camilamacedo86 force-pushed the test-asked branch 2 times, most recently from 78d26a6 to d71302b Compare February 26, 2026 17:38
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +1330 to +1332
// ResourceHasRestartAnnotation checks that a deployment has a restart annotation.
// This confirms that user changes stay in place after OLM runs.
func ResourceHasRestartAnnotation(ctx context.Context, deploymentName string) error {
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ResourceHasRestartAnnotation only supports Deployments (and the step text is deployment-specific). Renaming this to DeploymentHasRestartAnnotation (and updating the step binding) would make the intent clearer and avoid implying it works for arbitrary resources.

Suggested change
// ResourceHasRestartAnnotation checks that a deployment has a restart annotation.
// This confirms that user changes stay in place after OLM runs.
func ResourceHasRestartAnnotation(ctx context.Context, deploymentName string) error {
// DeploymentHasRestartAnnotation checks that a deployment has a restart annotation.
// This confirms that user changes stay in place after OLM runs.
func DeploymentHasRestartAnnotation(ctx context.Context, deploymentName string) error {

Copilot uses AI. Check for mistakes.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +37 to +39
# Wait for at least two OLM reconciliation cycles (controller runs every 10s)
And I wait for "30" seconds
# Verify user changes persisted after OLM reconciliation
Copy link

Copilot AI Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment hard-codes an assumed reconciliation period (“controller runs every 10s”). If reconciliation is event-driven (e.g., triggered by the rollout restart changing the Deployment), this note is misleading and the fixed 30s sleep may be unnecessary. Consider rewording to avoid a specific cadence and/or replacing the fixed sleep with waiting for an observable reconciliation signal.

Copilot uses AI. Check for mistakes.
Comment on lines +10 to +13
# TODO(Fix Boxcutter): Does not work with boxcutter yet.
# Blocked by https://github.com/operator-framework/operator-controller/pull/2515
@~BoxcutterRuntime
Scenario: User-initiated deployment changes persist after OLM reconciliation
Copy link

Copilot AI Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tag @~BoxcutterRuntime won’t be interpreted as “skip when BoxcutterRuntime is enabled”. CheckFeatureTags only skips when a scenario tag matches a known feature gate name and that gate is disabled; ~BoxcutterRuntime isn’t a known gate, so this scenario will still run under BoxcutterRuntime and may fail (per the TODO). Consider either removing this tag and relying on the test runner’s tag filters, or extending CheckFeatureTags to treat tags prefixed with ~ as “skip when the corresponding feature gate is enabled”.

Copilot uses AI. Check for mistakes.
Comment on lines +1331 to +1334
out, err := k8sClient("rollout", "restart", resourceName, "-n", sc.namespace)
if err != nil {
return fmt.Errorf("failed to rollout restart %s: %w", resourceName, err)
}
Copy link

Copilot AI Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On kubectl failure, this error drops useful diagnostics (stderr). Other steps include stderrOutput(err) in failures, which helps a lot when debugging CI flakes. Consider appending stderr (and possibly the command output) to this error message.

Copilot uses AI. Check for mistakes.
Comment on lines +1356 to +1359
// If the annotation exists and has a value, it stayed in place
if out == "" {
return false
}
Copy link

Copilot AI Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

k8sClient output may contain trailing newlines/whitespace; checking out == "" can be brittle. Elsewhere in this file empty kubectl output is checked with strings.TrimSpace(out) == "". Consider trimming before the emptiness check to avoid false negatives/positives.

Copilot uses AI. Check for mistakes.
Comment on lines +1341 to +1344
// ResourceHasRestartAnnotation checks that a deployment has a restart annotation.
// This confirms that user changes stay in place after OLM runs.
func ResourceHasRestartAnnotation(ctx context.Context, deploymentName string) error {
sc := scenarioCtx(ctx)
Copy link

Copilot AI Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step/function is deployment-specific, but the name ResourceHasRestartAnnotation reads like a generic resource assertion. Renaming it to something deployment-specific (e.g., DeploymentHasRestartAnnotation) would make failures and step registration clearer.

Copilot uses AI. Check for mistakes.
Comment on lines +1371 to +1373
// Note: We wait for a fixed time (not checking in a loop) because we need to make sure
// OLM actually runs (it runs every 10 seconds). We want to check that user changes stay
// AFTER OLM runs. If we just checked in a loop, we wouldn't know if OLM ran yet.
Copy link

Copilot AI Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comments here assume a fixed reconciliation cadence (“OLM actually runs every 10 seconds”), but the controllers appear largely event-driven and don’t generally requeue on a fixed interval. This can mislead future readers and encourages fixed sleeps. Consider rewording to avoid stating a fixed period, and (if possible) prefer waiting on an observable reconciliation signal rather than a fixed delay.

Copilot uses AI. Check for mistakes.
@camilamacedo86 camilamacedo86 changed the title 🌱 chore(ci): Add test to check if user changes are preserved WIP 🌱 chore(ci): Add test to check if user changes are preserved Mar 9, 2026
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 9, 2026
Add a test to ensure that OLM is not reverting user changes like kubectl rollout restart.

Assisted-by: Cursor/Claude
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants